In recent years, researchers have investigated detection of dynamic weather events (e.g., rain, snow and hail) in images and video sequences. The majority of investigated approaches focus on removal of weather events from the image sequences, or video sequences. These approaches may be categorized as de-noising methods, or restoration methods, since they consider rain (or snow) as a source of noise.
For example, Tripathi et al. (Tripathi, A. K. and Mukhopadhyay, S., “A probabilistic approach for detection and removal of rain from videos”, IETE Journal of Research, Vol. 57, No. 1, pp. 82-91, 2011) suggest that analyzing the symmetry of temporal variations in pixel intensity leads to distinct features for separating rain pixels from noise. Pixel temporal profiles affected by the presence of rain typically produce more symmetry than non-rain pixels (e.g., noise, objects). Also, the range of intensity fluctuations due to rain in a scene is much smaller than moving objects in the scene (e.g., traffic and pedestrians).
Wahab et al. (Wahab, M. H. A., Su, C. H., Zakaria, N. and Salam, R. A., “Review on Raindrop Detection and Removal in Weather Degraded Images”, IEEE International Conference on Computer Science and Information Technology (CSIT), pp. 82-88, 2013) review a variety of algorithms related to raindrop detection and removal from images. Their survey, however, is limited as they focus on removing raindrops from a car's windshield in order to improve driver visibility.
Park et al. (Park, W. J. and Lee, K. H., “Rain Removal Using Kalman Filter in Video”, IEEE International Conference on Smart Manufacturing Application, pp. 494-497, April 2008) introduce a rain removal algorithm using a Kalman Filter. As part of their approach, the authors estimate the intensity of pixels not affected by rain, thereby, restoring pixel values to their original intensity levels. Their approach models the intensity of each pixel with a Kalman Filter.
Wu et al. (Wu, Q., Zhang, W. and Vijaya Kumar, B. V. K, “Raindrop Detection and Removal Using Salient Visual Features”, IEEE International Conference on Image Processing (ICIP), pp. 941-944, 2012) suggest a method for raindrop detection and removal using visual features. Using a forward-looking vehicle mounted camera, their method seeks to remove raindrops from the acquired images. Their method assumes that individual raindrops are visible in the acquired images.
Chen and Chau (Chen, J. and Chau, L. P., “Rain Removal from Dynamic Scene Based on Motion Segmentation”, IEEE International Symposium on Circuits and Systems (ISCAS), pp. 2139-2142, 2013) describe a method for removing rain from dynamic scenes using motion segmentation. Photometric and chromatic properties of rain are used to detect the presence of rain, while motion segmentation is used to separate rain from other objects in the scene.
Wang et al. (Wang, D. J., Chen, T. H., Liau, H. S. and Chen, T. Y., “A DCT-Based Video Object Segmentation Algorithm for Rainy Situation Using Change Detection”, IEEE International Conference on Innovative Computing, Information and Control (ICICIC), 2006) develop a method for removing the effects of rain to improve object detection. Treating rain as a noise source, the authors attempt to remove the rain using a discrete cosine transform (DCT).
Xue et al. (Xue, X., Jin, X., Zhang, C. and Goto, S., “Motion Robust Rain Detection and Removal from Videos”, IEEE MMSP, pp. 170-174, 2012) suggest a method of rain detection and removal based on spatial and wavelet domain features. Their approach considers the edges of the raindrops and streaks as information, which is captured by using a wavelet decomposition.
Lui et al. (Liu, P., Xu, J., Liu, J. and Tang, X., “Pixel based Temporal Analysis Using Chromatic Property for Removing Rain from Videos”, Computer and Information Sciences, Vol. 2, No. 1, pp. 53-60, February 2009) suggest a rain removal technique based on temporal analysis and the chromatic property of rain. For detection, the authors segment the video into background and foreground regions. Rain pixels are determined by examining pixel-level differences between an input frame and its background.
Barnum et al. (Barnum, P. C., Narasimhan, S. and Kanade, T., “Analysis of Rain and Snow in Frequency Space”, International Journal on Computer Vision (Online), January 2009) suggest a model-based approach for analyzing dynamic weather conditions. Their approach models the effect of rain or snow in the frequency domain using the Fourier Transform.
Zhao et al. (Zhao, X., Liu, P., Liu, J. and Tang, X., “The Application of Histogram on Rain Detection in Video”, Proceedings of the 11th Joint Conference on Information Sciences, pp. 1-6, 2008) suggest a rain detection algorithm based on a K-means clustering method. Assuming a Gaussian Mixture Model (GMM) for the intensity histogram of each pixel, clusters are formed separating raindrops from other objects.
Bossu et al. (Bossu, J., Hautiere, N. and Tarel, J. P., “Rain or Snow Detection in Image Sequences Through Use of a Histogram of Orientation of Streaks”, International Journal on Computer Vision, Vol. 93, pp. 348-367, 2011) suggest a rain detection method based on segmenting objects into blobs. An assumption is made that rain streaks are visible within an image.
Hautière et al. (Hautière, N., Bossu, J., Biogorgne, E., Hilblot, N., Boubezoul, A., Lusetti, B. and Aubert, D., “Sensing the Visibility Range at Low Cost in the SafeSpot Roadside Unit”.) suggest a method for detecting dynamic weather events for vision-based traffic monitoring. Their approach suggests separating background and foreground regions in an image. The rain streaks are segmented from the foreground region by applying a gradient-oriented filter followed by a cumulative histogram. Rain or snow is detected by examining peaks in the histogram.
Finally, Tripathi et al. (Tripathi, A. K. and Mukhopadhyay, S., “Meteorological approach for detection and removal of rain from videos”, IET Computer Vision, Vol. 7, Issue 1, pp. 36-47, 2013) suggest an approach for detection and removal of rain based on meteorological properties of rain, such as shape, area, and aspect ratio of rain drops.
Conventional rain detection methods depend on detecting rain streaks in a video sequence captured by a camera. These methods pose a significant challenge when using low-resolution (spatial and temporal) CCTV (closed circuit television) surveillance cameras used in a traffic monitoring network. Shortcomings of the aforementioned methods include approaches that rely on an ability to adjust camera parameters and limit scene dynamics. In addition, most detection methods analyze an entire image (e.g., rain removal applications), under the assumption that rain is visible throughout an entire field-of-view. More importantly, there is an implicit assumption that these methods depend on high frame rate (greater than 20 fps) video sequences.
In general, many of the dynamic weather detection schemes concentrate on the appearance of rain streaks or snow streaks in the video. Assuming these features are visible, these methods employ time-domain or frequency domain filtering techniques to perform the detection. Model-based approaches are considered that produce analytical expressions for the rain or snow streaks. In addition, most of these methods are not suited for high dynamic environments or cluttered scenes that include moving traffic or other moving objects.